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Introduction 

This paper presents the legal iratlonale for using the National 
Teacher's Examination (NTE) validation procedure that was used in 
Arkansas to determine cut-scores fot Arkansas certlf ication. The 
paper also presents the actual directions given to certified 
practitioners from Arkansas public schools and faculty members 
trom CO lieges and universities in Arkansas who served aJ^Judges 
for the Arkansas NTE study of area examinations. The paper 
concludes with the data analysis procedures used in the Arkansas 
study « 

Legal Rationale 
This section of the paper presents the legal rationale for 
using the validation procedure that was used in Arkansas. In 
order to build the rationale, the NTE decision in South Carolina, 
Educational Testing Service (ETS) validation procedures and the 
Uniform Guidelines will be briefly reviewed. 

V . 

South Carolina Decision 

Oii* January 16, 1978, the United States Supreme Court announced 
it had summarily affirmed the April, 1977 decision of the Federal 
District Court, upholding Soiith Carolina's use of the NTE in the 
State's certification system. 

Five Justices affirmed the decision of the Federal District 
Court of South Carolina concerning use of the NTE for certification. 
They did not, however, issue an opinion for their reasoning. 
Justices White and Brennan did, however, record a strong dissent. 
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Furthermore, the case has not been cited as authority in any other 
Federal or State court opinion with regards to the use of NTE for 
certification purposes. ^ 

Some of the reasons the South Carolina Court used ?b enable 
the State to use the NTE were: 

1. There was no intent to discriminate against blacks 
iri? developing the testing system and that rather 
t;^han a racial classiilLcat ion, the system created 

a permissable classification based on knowlege or 
the lack of knowledge. 

2. The State's interest in hiring at least minimally 
competent teachers was related in a rational 
manner to its use of the NTE scores. 

3. The use of NTE scores was a valid "business 
necessity.*' This means that in some instances a 
discriminatory practice might be tolerated if it 
were essential to the operation ^of the business. 
The '^business necessity" rationale used by the 
South Carolina Court was based on a Supreme 
Court decision which said: 

The Supreme Court intended on examination 
of the alternatives available with respect to 
the legitimate employment objective identified 
by the employer to determine whether tRere is 
available to the employer an alternative 
practice that would achieve his business purposes 
equally well but with a lesser disparate Impact 
by race. In examining alternatives, the risk 
and cost to the employer are relevant. (Griggs 
V. Duke Power Co.) ^' 

In the United States v. South Carolina case, the Plaintiffs 

presented only one alternative, graduation from an approved teacher 

training program, to the use of the IjJTE for certification purposes. 

The trial Court did not feel that the alternative would achieve . 

the State's purpose in certifying minimally competent teachers aa 

well as the use of the NTE. The Court in support of this finding 



made two points. One, evidence demonstrated that the teacher 



training programs varied In admission requirements, academic standards, 

and grading practices. Two, evidence demonstrated that the State 

approves only general subject matter areas covered by the programs, 

not the actual course content of the programs. Both of these points 

would seem to weigh negatively on the Court's position that validation 

against the teacher training programs was sufficiently reflective of 

actual knowledge needed for the teaching positions. Here the Court 

would seem to be admitting that the twenty-five teacher training 

programs were In fact different and therefore not all would be to the 

same degree reflective of knowledge needed to competently perform 

the Job. The Court, however, while finding the teaching programs 

themBclvcs an Inadequate measure of teacher competency saw no 

inconH iHtcncy In finding test validation against those same teacher 

programH acceptable. 

ETS Validation Procedures 

ETS (1976) In their Brief Amicus Curiae filed In the District 

Court of South Carolina stated: 

Any general challenge by the plaintiffs to the use 
of the NTE by South Carolina for measuring the "academic 
preparation" of prospective teachers would be foreclosed 
by the North Carolina decision* There the three-Judge 
court held that: 

beyond argument . . • the State of North 
Carolina has the right to adopt academic 
requirements and written achievement tests 
designed and v^lldate^ to disclose the minimum 
amount of knowledge necessary to effective 
teaching. AOO F. Supp. at 348. (p. A) 

ETS (1976) describes the validation process as follows: 

The Content Review Panels were asked to make four 
Independent types of Judgments about the congruence of 
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the NTE tests they were reviewing and the curriculum in 
teacher training programs in South Carolina. Judges 
were asked to indicate, on the basis of a review of the 
major content topics of the test, whether the emphasis 
in the curriculum on each content topic was about the 
same as, more than, or less than the percentage of the 
'test devoted to the content topic. As an additional 
means of assessing the overall similarity between tests 
and curricula, judges were also asked to indicate one 
of four characterizations of greater or lesser similar- 
ity between the test and the curriculum of their 

institution. As a measure of the comprehensiveness of V 
the tests, judges were asked to indicate which, if any, 
major content topics in the teacher education sequence 
^ were not included among the test content topics. Finally, 
the judges were asked to review each question on the 
test to determine whether theN^dividual question was 
appropriate for use in South Carolina. The questions 
on the NTE were regarded a^s "content appropriate" if at 
least 51% of the judges indicated that at least 90% of 
the students taking the test would h^ve hafi an opportu- 
nity to learn the answer as part of their teacher 
education program or in courses prerequisite to enter- 
ing it. (pp. 17-18) 

This content validation process was upheld by courts in North 
Carolina and South Carolina when they ruled in favor of u^ng the 
NTE for certification in these States.' 
Unifonn Guidelines 

An important point concerning the South Carolina decision is 
that the decision was made before the Uniform Guidelines on Employ- 
ment Selection Procedures were adopted in 1978 by the Equal 
Employment Opportunity Commission, the Civil Service Commission, 
the Department of Labor, the Department of Justice and Treasury. 
These Guidelines are intended to assist private employees, government 
contractors, employment agencies, labor organizations, state and 
local governments, and the Federal Government in conplying with 
Federal law prohibiting discrimination by race, color, religion. 



sex, and national origin. Employment decisions such «• hiring , 



promotion, <lemotion, menvbership, referral, retention, licensing and 
certification are encompassed within -the scope of the Guidelines. 

The Uniform Guidelines do not require validation studies where 
a selection method does not adv^sely affect (An adverse effect is 
a selection rate for any racial, ethnic, or sex group which is less 
than 80Z of the rate for the highest scoring group.) a protected 
group. \^here validation is required, however, content , ^construct 
and criterion-related validity studies are equally acceptable, * 

Section 5, Uniform Guidelines on Emj)loyee Selection Procedures , 

General Standiyds for Validity Studies states: 

Evidence of the validity of a test or other 
selection procedure by a criterion related validity * 
Btudy should consist of empirical data demonstrating 
that the selection procedure is predictive of or 
8ir,nif icantly correlated with important elements of 
j ob performanc e. Evidence of the validity of a test 
or other selection procedure by a content validity 
study should consist of data showing that the selec- 
tion procedure is a representative sample of 
important work behavior to be perforirfed on the job 
for which the candidates are to evaluated. 
Evidence of the validity of a test or other selection 
procedure through a construct validity study should 
consist of data showing that the* procedure measures 
the degree to which candidates have identifiable 
characteristics which have been d€|.termined to be 
important is successful performanc^e in the job for ^ 
which the candidates are to be evatluated. 

The underlining was done by this writer to indicate that regard- 
less of the type of validity study one uses, the test one uses for 
certification has to be directly job related. Furthermore, Section 14, 
Uniform Guidelines on Employee Selection Procedures , Technical 
Standards for Content Validity states; 

Where a training program is used as a selection 
procedure and the content of a training program is 
justified on the basis of content validity, it should 



be justlf ledj^pn the relationship between the content of 
the tralninf^ program and the content of the job. 

In reference to the above guidelines section, Paul Tractenberg 

(1980), Professor ^of Law, Rutgers University states: 

The South Carolina NTE case does seem to suggest 
that content validity may be sufficient and does seem 
to suggest even that content validity may be sufficient 
if it is related to training programs; that is, the 
test items connect up to the training programs rather 
than to the job ofT the other end. That may be the only 
court decision that says really quite that, and as such, 
it is both a reasifrrance and a matter of some concern 
that it will become the uniform rule of the courts, 
especially in light of the new Uniform Guidelines on 
Employee Selection Procedures whiaj^ have a provision 
that seems to link content validity to what happens 
on tKc job, not simply what happened in the training 
program. • 

LoK<>l R/itionnlo Cor ArknnMnfi Procedures 

A paramount issue, in addition to measurement concerns, in the 
Arkansas NTE study was to make the procedures and the final results 
as legally defensible as possible. 

The project director, after consulting with several attorneys, 

believ^^that the ETS validation procedures used in the other ''NTE 

^^^^^^ ^ 
States** woulci not be upheld in a current court case. The primary 

reasons for this belief are the Uniform Guidelines on Employee 

Selection Procedures and court decisions made after the South 

Carolina NTE case. 

In Ensley v. Branch of N.A.A.C.P. v. Seibels, 616 F. 2d 812 

(5th Cir. 1980) there is a broad language indicating that validation 

to a training program is insufficient. The court says s 

c 

We do not believe the Davis rationale can be 
extended ... to the general proposition that any 
test can ba validated by showing a relationship to 
training. 
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We decline to extend the Davis rationale by holding 
that any test can be validated against training, without 
^^respect to the test's ability to predict job performance. 
Such an extension would violate the requirement of Job 
performance validation enunciation in Griggs and 
Abermarle as well as agency guidelines (Uniform Guide- 
lines) elaborating upon that requirement, 

A case whit* explicitly says that the Equal Employment Oppprtu- 

nity Commission (EEOC) guidelines should apply is United States v, 

Virginia, 62(/F>2d 1018 (4th Cir, 1980) , In Virginia the state was 

giving a test for employment of state police officers. The state 

had validated the test according to the content of the training 

programs for the job. The court said: ^ 

In Davis the court held that a positive relation- 
ship between the test and training-course performance 
was a sufficient validation under standards similaif 
to those of Title VII. While the evidence presented 
to the district court was sufficient to support its 
conclusion under Davis , upon demand the court should 

reconsider this issue in the light of the applicable ^ • v ' ^ ' 

EEOC guidelines to determine whether a contrary V\ ^ 

conclusion is required. 

The reader should realize that the Davis case was used by the 
South Carolina Court in their NTE decision. Furthermore, the Davis 
case was also rendered before the Un:{fQrm Guidelines were pubrished. 

The Arkansas NTE validation procedure was designed to meet the 
intent of the general validity standards of th^ Uniform Guidelines . 
In other words, the validity question was designed to be job related, 
instead of training program related. 

Any state using the NTE may, however have legal problems, 
regardless of the validation procedures. A recent test validation 
case is Guardians Association of the New York City Police Department, / 
Inc. V. Civil Service Commlaslon. 630 F. 2d 79 (1980 ). The court 
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in this case stated five steps for te^ validation. They were: 

1. The test-makers must have conducted a suitable 
job analysis; 

2. they must have used reasonable competence in 
constructing the test itself; 

3. the content of the test must be related to the 
content of the job; 

4. the content of the tegt must be representative 
of the content of the job; and 

5. a scoring system must be used which usefully 
selects from among the applicants those who 

can better perform the job. i ^ ^ 

Directions to the Judges 
The actual directions used by the judges In the Arkansas study 

were: 

The first rating you will make concerns item Relevance. 
This will be used for test validation. In order to 
make thj.8 judgment, you should read the item, the 
•'correct*' answer, and the distractors. (The correct 
^ ahswer is underlined in the tefit booklet.) You should 
then judge the relevance of the content measured by the 
question with respect to the domain of knowledge you 
believe a minimally qualified entry-level person in 
the certification area should ^^possess . 

If you believe the content of the question is irrele- 
♦ vwnt to the domain of knowledge a minimally qualified 

entry-level person in this field should possess, then * 
you should fill-in circle 1 on your answer sheet in 
the Relevance column to signify "Not Relevant." 



If you believe the content of the question is of doubt- 
ful or questionable relevance to the domain of knowledge 
a minimally qualified entry-level person in this field 
should possess, then you should fill-in circle 2 on 
your answer sheet in the Relevance column to signify 
"Questionable." 

If you believe the content of the quMtion is important, 
but not quite crucial, to the domain of knowledge a 
minimally qualified entry-level person in this field 
should possess, then you should fill-in circle 3 on 
your answer sheet in the Relevance column to signify 
"Important. j" 
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If you believe the content of the question Is of crucial 
importance to the domain of knowledp,e a minimally 
qualified entry-level person in this field should possess, 
then you should fill-in circle A oh your answer sheet in 
the Relevance column to signify ''Crucial. 

Points to consider for the relevance ratings: 

A. A minimally qualified entry-level person will have 
completed the training requirements as specified 
by Arkansas Certification Requirements. The 
requirements were sent to you for your NTE area. 

B. A minimally qualified entry-level person has the 
minimal amount of knowledge, to initially perform 
in the certification area. 

C. In making your relevatice judgment, hp sur^ to 
think in terms of the full range of grade levels 
and/or subject areas and/or types of schools for 
your certification. Examples: 

a. If you are judging the NTE Elementary Educa- 
tion for Elementary (1-6) Certification and 
you find an item "Not Relevant" for early 
elementary grade teachers, but "Crucial" 

for later elementary grade teachers, then 
you would mark the item "Crucial." 

b. .If you are judging the NTE Math examination 
for Math Certification and you find an item 

, "Questionable" for teaching 9th or 10th grade 
math, but "Important" for teaching 11th or 
12th grade math, then you would mark the 
item "Important." 



If you are judging for the Elementary 
Principal Certification and find an item 
"Crucial" for a large urban elementary 
school, but "Questionable" for a small 
town elementary school, then you should 
mark the item "Crucial." 

If you are judging the NTE in Biology anc 
General Science, then you should respond) 
for Biology or General Science Ccrrtifici 
If y6u are Judging the NTE in Chemistry 
Physics, then you should respond for Chi 
or Physics o£ Physical Science Certification. 
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The second jujfigTnent you will make about each item will 
help determine the cut-score. You should imagine a 
hypothetical person, who in your, judgment , has the 
minimum amount of academic knowledge to complete the 
preparation pr'bgram required for certification in 
Arkansas and has the minimum amount of knowledge to 
perform in the field designated by the NTE area test. 
VJith this hypothetical person in mind, you are to 
estimate the probability that this minimally competent 
person would know the answer to the NTE item without 
guessing. Another way of thinking about this estimation 
process is to think of a group of minimally competent 
persons and then estimate the percent of minimally 
competent persons who would answer the NTE item 
icorrectly without guessing. 

Before you make your estimate about the item, you 
should also realize the item difficulty based oit the 
NTE nom group for the item. The item difficulty or 
the percent who have passed the item is written beside 
the itan in* the booklet. ^ 

You should mark your estinvate for each item on the 
rof^ponse sheet under the I*robability column. You 
should use the following flfcale for these estimates: 



Fill 


in 


circle 


1, 


if 


you r 


estimate 


is 


between 


.00 




.10 


Fill 
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if 
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Data Analysis 

T\\iQ section of the paper presents the methods used to determine 
item validity and cut-scores. The attached chart presents the actual 
results from the Arkansas study. 
Validity 

The validity of each Item was determined by computing «n item 
mean for each itenTon the relevance scale. This scale had a range ' 
from one (Not Relevant) to four (Crucial) . In order for an item 
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to be considered valid mean score on the relevance scale had 

to be gr&ater than, 2.5. In other words, the Item had to be rated 
by th^ Judges as closep: to the Important category than to the 



questionable category. If half of the judges had rated the Item 
questionable and the other ^alf had rated the Item important, then 
the Item would not have met the validity criterion since the mean v 
rating would have been ^.50. 

Cut-Scores - k 

, The cut-scores for each area examination was determined by a 
slight modification of a procedure knowa as the Angoff method. The 

first step for detemlnlng the cut-score was to determine an item 

J' 

mean on the probability scale. Since the Judges had responded to 
a probability range for each Item, the mld-polnt of *the range was 
used to' compute the Item mean. For example, a one.^n the probability 
scale was converted to .05 since one represented the probability 
between .0 through .1. / 

The raw score cut-score for each area examination was computed 
by summing the mean probabilities for only the items that had met 
the validity criterion. A conversion formula was then used to 
convert the raw scores to NTE standard scores. 
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Chart 





Number 


Number 


NTE Area 


Not Valid 


Valid 


Art Education 


55 


95 


Biology and General Science 


19 


141 


Business Education 


35 


125 


Chemistry, Physics, and General 






Science 


29 


121 


Early Childhood Education 


28 


122 


Education in the ElemjBntary School 


. 21 


129 


English Language and Literature 


42 


108 


French 


55 


125 


Guidance Counselor 


8 


147 


Home Economics Education 


26 


124 


Industrial Arts Education 


44 


. 106 


Introduction to the Teaching of 






Reading 


12 


138 


Mathematics J^athematlcs) 


22 


98 . 


Mathematics (Basic Math) 






Media Specialist-Library and 


24 


126 


Audiovisual Sjervlce 


Music Education 


30 


120 


Physical Education 


31 


119 


Reading Specialist 


16 


134 


Social Studies 


36 


114 


Spanish^ 


44 


116 


Speech Pathology 


32 


118 


Educational Administration and 






Supervision (Elementary School 






Principal) 


38 


112 


Educational Administration and 






Supervision (Secondary School 






Principal) 


44 


105 


Educational Administration and 






Supervision (Adimlnlstrator) 


27 


123 
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Arkansas * . 

Derived ' . Certification • 

Cut-Score Percentiles Cut-Score 'Percentiles 

500 19^ - . 450" - 8 

580 29 540 17 

590 44 550 28 

. 560 - 41 520 26 

560 32 500 18 

550 28 500 16 

530 ' 26 490 15 

530 29 . 490 . 19 

.610 39 560 23 

570 30 520 16 

^80^ ■ 31 550 23 



■560^,,^ 28 510 16 



570'" 46 530 30 

520 26 470 11 

590 19 540 10 

570 *>6 510 18 

590 40 540 22 

600 32 550 17 

550 36 . 500 . 20 

520 23 470 11 

630 29 570 11 

540 19 490 9 

560 25 510 12 

570 27 520 15 
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NTE Sample Items 
Education In the Elementary School 



A teacher Is most likely to be effective In encouraging children 
to Improve their handwriting by having them do which of the 
following? 

(A) Work on the letters that are clrcl^^d on their papem. 
CbS Keep samples of their handwriting for comparison 
purposes. 

'(C) Practice the basic ovals and slants for the difficult 
letters. 

(D) Read each other's papers and Identify Illegible letters. 

(E) Recopy written work In their best handwriting. 

An Intermedlate^grade teacher who wants to develop pupils'' Interest 
In poetry could most profitably use poems In th« shorty humorou* 

style of * ' 



Edward Lear 
James Whltcomb Riley 

(C) Henry Wadsworth Longfellow 

(D) Rachel Field 

(E) Rudyard Kipling 



In which of the following ways may a teacher best provide tor the 
development of reading dkllls In an upper elementary class of 
thirty pupils whose reading abilities range from second to eighth 
grade? 

(A) By grouping the children Into three, levels of basic readers 

(B) By arranging for those pupils who are below grade level to 

go Into a lower grade for reading Instruction 

(C) By grouping the children Into six levels of basic readers 

(D) By providing Instruction In the sixth-grade reader for all 

children 

^e) By placing the children Into skill groups and usln^ 
Individualized approach 

A pupil has, become proficient In Independent word attack when he 
has learned which of the following: 

(A) The phonics principals presented In the basal reader 
CB% How to use a decoding system that Is helpful 

(C) How to use context clues In analyzing unknown words 

(D) The sound-symbol relationships of the English language 

(E) How to recognize words from configuration clues 
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5. .Engaging in creative art activity is likely To increase"" a child's 
appreciation of^art because 

the process of creation sensitizes him to many aspects 
of the completed product 

(B) every activity in art generates so many overtones that 

appreciation is inevitable 

(C) reference to actual examples helps the child in his 

creative work 

(D) the physical activity involved sharpens perception 

(E) creation and appreciaticTn are fundamentally the same 

6. After dinner,' 3/4 of the pie was left. Mary then ate 1/2 pf what 
was left. How much did she eat? ^ 

Which of the following mathematics operations is used to solve 
the problem ab^ove? 

(A) Addition of fractional numl>ers ' , 

(B) Division of fractional numbers* * ' 

(C) Division of whole numbers 

(iT) Multipli cation of fractional numbers 
(E) Subtraction fractional numbers 

7. After completing a unit on Native Americans, an intermediate-grade 
teacher asks his class to indicate on a topographical jmap of a 
mythical country where various tribes might have lived. His 
objective prol^ably is to • ' • 

(A) help the children build theories about peoples and their 

housing paftterns 

(B) see whether the children know how to read a map 

(C) see whether the children have learned where the Native 

Americans lived 
see whether the children can match modes of adaptation 
to environmental areas 
(E) help the children develop more accurate map skills 

8. A middle-grade teacher who wishes to update and Improve her teach- 
ing of science is most likely to do so by 

(A) placing more stress on reading in the basic science texts 

(B) including more lecturies to save time for experiments 
guiding children to discover leaming^s for themselves 

(D) demonstrating more experiments 

(E) requiring children to write up experiments in greater 

detail 
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9. Music can be effectively related to social studies for which 
of the following reasons? 



^A^ Music has a pl^ce in the cultural and nationalistic 
heritage of all nations. 

(B) Folk music is a reliable source of historical data 

among nations. / 

(C) Music is the best source of information about a 

people's culture. 

(D) All nations have produced working and communal songs 

that can be used in the schools.-^ 

(E) Nationalistic differences in music cori^espond to 

political differences among nations. 

10. According to usual patterns of child growth and development^ most 
6-ycar-oldB could be expected -to be able %o do which of the 
following? 

(A) Jump rope rhythmically 

(B) Tlirow, catch, bounce, and bat a ball 

(C) Roller-skate and ride a bicycle 

(D) Climb a rope ^ 

Hqp on one foot, skip, and Jump 
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Rel. 
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4 


Prob . 
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.15 
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.35 
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.55 


.65 


.75 .85 
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